The present disclosure relates generally to calibration of 3D sensors and, more particularly, to 3D sensor installation in a passenger conveyance applications.
Installation of a passenger conveyance such as an elevator system may incur considerable expense and inconvenience in labor, time, test fixtures, measurement devices, and data entry. This may be exacerbated when one or more 3D depth sensors are used for additional functionality such as determining passenger expected time of arrival (ETA).
3D depth sensing systems typically employ a sensor-centric coordinate system wherein one dimension (depth) is radially away from the sensor and two other dimensions (u,v) are orthogonal to each other and perpendicular to the depth dimension corresponding to the center of the (u,v) plane. This sensor-centric system is often not in metric units, and is often not aligned with a world coordinate system (x,y,z) where x, y, and z are mutually orthogonal, the (x,y) plane is the floor of a building or structure, and the z dimension is vertical. In this context, metric units are not limited to those of the International System of Units (SI), also known as the Metric System, but denote any mensuration units readily convertible to Metric System units. Similarly, a world coordinate system is any coordinate system different from an inherent sensor-centric coordinate system and is often a Cartesian coordinate system.
Calibration of installed 3D depth sensing systems to world coordinates in metric units might employ 4 independent locations at different, independent, measured, (x,y,z) world coordinates. The 4 locations might be of ‘found’ objects that happen to be in the Field of View (FoV). However, for a professional installation, a portable 3D calibration jig might be carefully emplaced at measured world coordinates and calibration data of this jig taken by the 3D depth sensor. With data of the world coordinates (xi,yi,zi), and the corresponding 3D depth sensor observed coordinates, (ui,vi,di), i=1, . . . 4, a transform matrix can be solved to achieve calibration. More than 4 points may be measured for a more robust solution using least squares techniques.
Conventional 2D systems that use a single image sensor do not have enough information to map sensed objects to a world coordinate system (invert the 3D to 2D projection). Such systems might utilize multiple views of some moving object to compute calibration parameters, but this calibration is ambiguous in scale since there is no absolute size reference. This approach is further complicated in that some moving objects move in a non-uniform manner and deform during movement.
Individual sensor calibration typically consists of determining the 6 extrinsic sensor parameters that are used in the computation of a transform matrix (sometimes also called a projection, translation and rotation matrices, or their inverses). The 6 extrinsic parameters are x, y, z offsets from the origin of a world coordinate system and the pitch, yaw, and roll angles with respect to the world coordinate axes. A complete calibration might include determining both the 6 extrinsic parameters and an additional 5 intrinsic parameters. As is well-known, the 5 intrinsic parameters are fx and fy (the focal lengths), x0 and y0 (the principle point offsets), and s (the axis skew). This can be thought of as a 2D translation, 2D shear, and 2D scaling. The intrinsic parameters are typically known, e.g., from manufacturer's data, or may be determined by known techniques prior to installation of the sensor.
However, there has been relatively little experience with 3D sensor automatic calibration, particularly where some or all of the extrinsic calibration parameters are known a priori. Conventionally, a tedious calibration approach is taught to manually place a number of detectable objects in the 3D sensor's Field of View (FOV), maintain the objects stationary at their locations, and measure the locations in both world coordinates and in sensor coordinates. This procedure may be simplified using a special 3D calibration target or jig of known dimensions.
With a sufficient number of unique measurements, a mathematical equation may be solved to determine the transform matrix. Often more than the minimum number of measurements are used (at the expense of even further effort and possible error) to compute a more robust solution.
The mathematical equation may be formulated asTxs=xw 
Where xs=(u,v,d) is a point's coordinates in the sensor coordinate system, xw=(x,y,z) is the same point's coordinates in the world coordinate system, and T is the desired transformation matrix, which can be expressed in homogeneous coordinates as:
  T  =      [                                        m            11                                                m            12                                                m            13                                                m            14                                                            m            21                                                m            22                                                m            23                                                m            24                                                            m            31                                                m            32                                                m            33                                                m            34                                                0                          0                          0                          1                      ]  
This may be reformulated asA*M=B  (1)
where
      A    =          [                                                  c              11                                                          c              12                                                          c              13                                            1                                0                                0                                0                                0                                0                                0                                0                                0                                                0                                0                                0                                0                                              c              11                                                          c              12                                                          c              13                                            1                                0                                0                                0                                0                                                0                                0                                0                                0                                0                                0                                0                                0                                              c              11                                                          c              12                                                          c              13                                            1                                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                ⋮                                                              c                              n                ⁢                                                                  ⁢                1                                                                        c                              n                ⁢                                                                  ⁢                2                                                                        c                              n                ⁢                                                                  ⁢                3                                                          1                                0                                0                                0                                0                                0                                0                                0                                0                                                0                                0                                0                                0                                              c                              n                ⁢                                                                  ⁢                1                                                                        c                              n                ⁢                                                                  ⁢                2                                                                        c                              n                ⁢                                                                  ⁢                3                                                          1                                0                                0                                0                                0                                                0                                0                                0                                0                                0                                0                                0                                0                                              c                              n                ⁢                                                                  ⁢                1                                                                        c                              n                ⁢                                                                  ⁢                2                                                                        c                              n                ⁢                                                                  ⁢                3                                                          1                              ]            M    =                            [                                                                      m                  11                                                                                    ⋮                                                                                      m                  14                                                                                                      m                  21                                                                                    ⋮                                                                                      m                  24                                                                                                      m                  31                                                                                    ⋮                                                                                      m                  34                                                              ]                ⁢                                  ⁢        B            =              [                                                            x                1                                                                                        y                1                                                                                        z                1                                                                        ⋮                                                                          x                n                                                                                        y                n                                                                                        z                n                                                    ]            
It should be understood that a transform matrix or, equivalently, a calibration matrix, is a convenient, concise representation of the consistent relationship between a point in one coordinate system and the same point in another coordinate system. The calibration taught herein is not restricted by this matrix representation and other equivalent geometric and analytical representations are disclosed herein to illustrate this.
Mϵ12×1 are the first 12 elements of T in a column, Aϵ3n×12 contains the projection of the n measured points in the sensor coordinates into the world coordinate system, and Bϵ3n×1 contain the coordinates of n points in world coordinates. Here, the cij are the projection of the sensor image and depth (u,v,d) to sensor coordinates via the sensor intrinsic parameters. Solving eq. (1) for M by least squares produces the values for the desired transform matrix.
Although effective, special calibration targets and significant installer effort may be required by present methods.